Skip to content

docs+fix: say what is actually guaranteed (#598 contract, #599 relabel) - #607

Merged
Taleef7 merged 8 commits into
mainfrom
fix/say-what-is-guaranteed
Sep 21, 2026
Merged

Taleef7 merged 8 commits into
mainfrom
fix/say-what-is-guaranteed

Conversation

@Taleef7

@Taleef7 Taleef7 commented Sep 21, 2026

Copy link
Copy Markdown
Owner

Closes #599. Addresses the documentation half of #598 and leaves that issue open for the
primitive, with the reason stated below. Both findings are from the 2026-09-07 review. Two commits,
one per issue.

Neither is a behaviour change. Both are a claim catching up with the code.

#598 — "every state change writes an audit_event" is true of operator actions, and not of runs

CLAUDE.md states the rule with "no exceptions". The run path is an exception, and it was
discoverable only by reading run-pipeline.ts — which is exactly the shape of claim
DATA_MODEL_CONTRACTS exists to stop.

The two paths make opposite trades on purpose, and both stay:

order what a failure between the two leaves
operator action (case/case-actions.ts) event then patch an action recorded but not applied — never an unaudited state change
run (run/run-pipeline.ts) upsert then best-effort audit a state change applied but unaudited — the violation

The run's ordering is deliberate: the alternative strands an otherwise-complete run as RUNNING after
the case was already mutated. recordCaseEvent already makes the action row and the audit row one
transaction, and bulk assign takes the operator side too, auditing before it mutates.

Why the primitive is not in this PR

applyCaseAction({ patch, action, audit }) cannot live in one store. The action and audit rows
belong to CaseEventStore — which already opens its own BEGIN/COMMIT — while the patch belongs
to CaseStore. Making all three one unit needs a transaction seam spanning both stores, which is an
architectural change rather than a refactor, and it wants local Postgres to verify: the SQLite
floor cannot catch Pg-only SQL, and both #544 defects passed on it.

So #598 stays open for that, and the contract now carries the operative line: until it exists,
nothing should build operational reliance on the ledger being complete for run-created transitions.

A reconciliation job is not a substitute — without durable operation identity, an expected version, a
deadline and a visible failure state it is a second unreliable thing checking the first.

The correction lands in both always-loaded places: DATA_MODEL_CONTRACTS §4 beside the
best-effort note, and CLAUDE.md's rule itself. A rule whose exception lives in a source file is a
rule a session will contradict.

#599 — an approval gate that could not fail on the thing its label implied

Studio rendered "Test Fixtures ✅" and blocked activation until it passed, which reads as the
fixtures ran and the measure produced the expected outcomes
.

validateTests never executes anything. It checks the list is non-empty and that each entry has a
name, a subject, and an expectedOutcome in the allowed set. A fixture asserting an impossible
outcome passes, and so does one that contradicts the CQL.

The row now reads "Fixtures Well-Formed — present and well-formed; not executed against the
measure"
, and the function's docstring says the same.

The guard went on the semantics, not the copy

A render test asserting label text would need four child panels mocked to check a string, and would
pin the wording rather than the meaning. Instead three backend tests pin the limitation:

  • a fixture naming a subject that exists in no directory and no corpus still passes;
  • two fixtures asserting opposite outcomes for the same subject both pass;
  • and the other half of an honest label — what it genuinely does check — is pinned too, including
    that each failure names which fixture.

If someone implements execution (#599's option 2, the real fix), those tests fail — which forces
the label and the behaviour to move together instead of drifting apart again.

Relabel only, and the blast radius was bounded: Studio authoring is hidden from the pilot's
CASE_MANAGER seats, so the misleading tick was in front of engineering users only. That is why it
was not urgent, not a reason it was fine.

Verification

new backend tests 3, all passing
measure tests 9 / 9
tsc --noEmit clean, both projects
npm run lint clean (2 pre-existing warnings in untouched files)

The full backend suite ran green on this branch's base (2,838 tests — 2,814 pass, 23 skip, 1 local
.official-content failure unrelated to these files).

Taleef added 7 commits September 21, 2026 11:24
…porter says what the document says

#594. The 2026-09-07 review's one high-priority correctness defect, accepted
2026-09-08 and untouched since.

prepareForQiCore filled four coded fields when it could not bind them:
Condition.clinicalStatus, verificationStatus, category and Encounter.class.
Its guard, unbindable(), is true of a MISSING field as well as a
present-but-unbindable one, and both branches assigned a module default.

Two consequences. First, an absent field was invented — live rather than
latent, because the QRDA-I import emits none of those three on a Condition
and no class on an Encounter, so preparation minted them on a third party's
document, including stamping `active` on a Condition the importer had just
given an abatementDateTime from a closed interval.

Second, and worse than the issue recorded: a PRESENT code was discarded.
unbindable() is as true of a system-less `resolved` as of a system-less
`active`, so a corrected misdiagnosis was reported as an active, confirmed
problem — that patient enters CMS122's denominator and, with no HbA1c, its
numerator. The file's own docstring claimed this hole was closed.

So: this layer supplies a SYSTEM, never a CODE. Normalize only when a value
is present, cannot bind, and carries a code from that field's own value set,
writing that same code back. Absent stays absent; bindable is untouched; an
unrecognised code is left alone, because we cannot claim to know which
system it came from.

The mapping moves to where the source semantics are known. times() now
reports three states rather than two, because collapsing them made a
faithful mapping impossible however it was written: a <high> with a value
closes the interval (resolved, beside its own abatementDateTime), a <high
nullFlavor="UNK"/> is QDM open prevalence (active), and an absent <high> is
silence and emits nothing.

Nothing the pilot reports moves: the ADR-075 corpus records all four fields
itself, fully systemed, with category distinguishing an encounter diagnosis
from a problem-list item — and cms122/125/2/137 still find real populations.

Fixtures are adversarial per the review's bar: refuted, resolved,
entered-in-error, an unrecognised code, an inpatient class, and the three
interval shapes. Four mutations killed, including "code discarded, default
substituted" — the first cut of this fix turned Encounter {code:"IMP"} into
ambulatory and an existing test caught it.
#595, and ADR-086 covering it with #594.

A corpus bundle built as of 2026-09-07 carried 19 future-dated clinical
events in the first 48 records — a PHQ-9 on 8 October, a blood pressure on
14 October, mammography in October and November. The bundle knew things that
had not happened.

Facts are generated for the calendar year the evaluation date falls in,
which is correct (ADR-072 scores a calendar year), and nothing then filtered
what was EMITTED by the as-of.

The cutoff filters on the date each fact was RECORDED — the value already
handed to provenanceFor — and not on "every date inside the resource is in
the past": a medication order known today may legitimately carry a future
intended end, and dropping it would be a different wrong answer. A resource
and its Provenance are emitted together, so a filtered fact leaves no
status, abatement, reference or provenance behind.

No reported number moves, and that is asserted rather than argued: every
official measurement is taken at 31 December, so the year-end cutoff is
compared against an unbounded one and must be identical. cms122, cms125,
cms2 and cms137 still find real populations. A separate test requires the
mid-year cutoff to actually remove something, so the no-foreknowledge sweep
cannot pass against a filter that does nothing.

It stops being invisible the moment anything evaluates at another date: a
mid-year rerun, a demo "as of today", an encounter-time evaluation (MM-4),
or an acceptance cohort built around a timing boundary.
…entry keeps its neighbours

Both Codex findings on #606, and both are the same class as the defect the
PR removes — which is the part worth recording.

A malformed `<high value="20240230"/>` was read as an open interval. The
first cut asked "a <high> exists and produced no date", which is equally
true of a date the SOURCE asserted and this importer could not parse. It
reported `active`: a status the document never made, about a condition
whose end we simply failed to read, and one that can put the patient into a
measure population. It is now keyed on the nullFlavor attribute, and on any
spelling — NI, NA and ASKU all mean the source addressed the end and
recorded no value for it, so keying on UNK alone would read the rest as
silence. A parse failure is a third thing and says nothing.

Normalizing one `category` entry discarded the others. The first cut
flattened every entry's codings, picked one recognised code and assigned
the result AS the whole array, so a Condition carrying two categories kept
one and lost the other with its text and extensions. Each entry is now
normalized on its own, everything else it carries is preserved, and an
entry that cannot be normalized passes through unchanged — the same reason
an unrecognised code is left alone one level up.

Three separate cuts of this change have now re-committed the error it
exists to remove: substituting a default code for IMP, reading a parse
failure as an assertion, and replacing an array to normalize one of its
entries. Each was caught by a test rather than by review of the idea, which
is the argument for writing the adversarial fixture first. ADR-086 records
it.

Five tests added, three more mutations killed.
…un path (#598)

CLAUDE.md states "every state change writes audit_event — no exceptions".
The run path is an exception, and it was discoverable only by reading
run-pipeline.ts, which is the shape of claim DATA_MODEL_CONTRACTS exists to
stop.

The two paths make opposite trades on purpose and both stay. An operator
action records the event FIRST and then applies the patch (recordCaseEvent
makes the action row and the audit row one transaction), so a failure
between them leaves an action recorded but not applied — never an unaudited
state change. A run upserts the case FIRST and audits best-effort, so a
failure there leaves a state change applied but unaudited, accepted because
the alternative strands an otherwise-complete run as RUNNING after the case
was already mutated.

What is missing is the primitive, not the ordering — and it cannot live in
one store: the action and audit rows belong to CaseEventStore, which already
opens its own BEGIN/COMMIT, while the patch belongs to CaseStore, so
applyCaseAction({patch, action, audit}) needs a transaction seam spanning
both. Deferred deliberately: it wants local Postgres, since the SQLite floor
cannot catch Pg-only SQL. Until it exists, nothing should build operational
reliance on the ledger being complete for run-created transitions, and a
reconciliation job is not a substitute without durable operation identity,
an expected version, a deadline and a visible failure state.

Also #599's backend half: validateTests checks SHAPE, not outcomes, and its
docstring said so only by omission. Three tests pin the limitation rather
than the copy — a fixture naming a subject that exists nowhere passes, and
two fixtures asserting opposite outcomes for one subject both pass. When
execution lands, those tests fail, which forces the label and the behaviour
to move together.
#599)

Studio rendered "Test Fixtures ✅" and blocked activation until it passed,
which reads as: the fixtures ran and the measure produced the expected
outcomes.

validateTests never executes anything. It checks the list is non-empty and
that each entry has a name, a subject, and an expectedOutcome in the allowed
set — so activation was gated by a control that could not fail on the thing
its label implied, which is the vacuous-guard shape this repo keeps finding.

The row now reads "Fixtures Well-Formed — present and well-formed; not
executed against the measure".

Relabel only. Executing them is the real fix and belongs with the next
Studio work; #599 carries it, and the backend tests in the previous commit
are what make the two move together — they pin the limitation, so
implementing execution fails them.

Blast radius was bounded: Studio authoring is hidden from the pilot's
CASE_MANAGER seats, so the misleading tick was in front of engineering users
only. That is why it was not urgent, not a reason it was fine.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b236718703

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread CLAUDE.md Outdated
Comment on lines +33 to +34
transition audits best-effort AFTER the upsert, so it can mutate and lose the event. Operator actions
audit first and cannot. `DATA_MODEL_CONTRACTS` §4 carries which is which and why.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Scope the operator-audit guarantee to case actions

When an operator creates, approves, activates, or deprecates a measure and the subsequent audit write fails, measure-lifecycle.ts has already called createMeasure or setVersionStatus (lines 54–55, 69–70, 87–88, and 118–119), leaving an unaudited operator state change. Therefore this newly added always-loaded claim that operator actions universally audit first and cannot lose an event—and that #598 is the one known gap—is false outside the case-action paths; qualify it to case operator transitions or document the additional mutation-before-audit paths.

AGENTS.md reference: AGENTS.md:L5-L9

Useful? React with 👍 / 👎.

…er than recalled

Codex on #607. The first cut said "operator actions audit first and cannot
lose the event": true of CASE actions, false of several other operator
surfaces. That is the same failure one level up, inside a change whose
entire subject is claims being wider than the thing behind them.

Enumerated by sweeping for the mutate-before-audit shape instead of
recalling it.

AUDIT FIRST, cannot produce an unaudited state change: every case action
(recordCaseEvent makes the action row and the audit row one transaction,
patch follows), rerun-to-verify's case patch, bulk assign and panel
backfill via the batch form.

MUTATE FIRST, can apply a change and lose the event: the run-created case
transition; the measure lifecycle (create :54, approve :69, deprecate :87,
transition :118); segment create (routes/segments.ts:172);
terminology-mapping create (value-set-governance.ts:335).

Only the run's ordering is a considered trade — the alternative strands an
otherwise-complete run as RUNNING after the case was already mutated. The
other six are simply the order they were written in, which splits #598 into
a cheap half (flip them; no seam needed) and the primitive that still wants
a cross-store transaction. Recorded on the issue.

case-rerun.ts looked like a violation and is not: its first mutation creates
a RUN row, and the case patch is explicitly after an audit-first
recordCaseEvent. Checked rather than assumed, in both directions.

The sweep covered admin, case, measure, quality, program, compliance and
routes and is a heuristic, so the list is what it found rather than proof of
completeness — stated as such.
@Taleef7

Taleef7 commented Sep 21, 2026

Copy link
Copy Markdown
Owner Author

Review response — 3b54eafa

You are right, and the finding is sharper than a scoping nit: my correction to an over-broad claim
was itself over-broad
, in a change whose entire subject is claims being wider than the thing behind
them. Same failure, one level up.

Rather than patch the one path you named, I swept for the mutate-before-audit shape across
admin, case, measure, quality, program, compliance and routes, so the list comes from
the code rather than from memory.

Audit first — cannot produce an unaudited state change:

  • every case action (case/case-actions.ts) — recordCaseEvent makes the action row and the audit
    row one transaction, and the patch follows
  • rerun-to-verify's case patch (case/case-rerun.ts)
  • bulk assign and panel backfill, via the batch recordCaseEvents

Mutate first — can apply a change and lose the event:

path sites
run-created case transition best-effort audit after the upsert
measure lifecycle createMeasure :54, approveMeasure :69, deprecateMeasure :87, transitionStatus :118 — the four you found
segment create routes/segments.ts:172
terminology-mapping create measure/value-set-governance.ts:335

One correction in the other direction, since the sweep is a heuristic and I checked its hits:
case-rerun.ts looked like a violation and is not. Its first mutation creates a run row, and the
case patch is explicitly after an audit-first recordCaseEvent with a comment saying so. It would
have been easy to assert that one wrongly while fixing the others.

What this changes about the work

Only the run's ordering is a considered trade — the alternative strands an otherwise-complete run
as RUNNING after the case was already mutated. The other six are simply the order they were written
in. That splits #598 into two separable pieces, which is now recorded on the issue:

  1. flip the six to audit-first — cheap, no seam, strictly the safer side, removes most of the
    exposure;
  2. the primitive, which still needs a transaction seam spanning CaseEventStore and CaseStore
    and wants local Postgres to verify.

CLAUDE.md now says the rule is the RULE and names which paths do not yet meet it, with "write new
code audit-first"; DATA_MODEL_CONTRACTS §4 carries the enumerated list and states that the sweep is
what it found rather than proof of completeness.

@Taleef7
Taleef7 merged commit acc4b76 into main Sep 21, 2026
29 checks passed
@Taleef7
Taleef7 deleted the fix/say-what-is-guaranteed branch September 21, 2026 16:09
Taleef7 added a commit that referenced this pull request Sep 21, 2026
…e gets a test (#612)

* fix(audit): finish #598's triage — five more audit-first, and the rule gets a test

The owner's #598 decision was that a path which CAN audit before it mutates should. #607/#608 took the
plain reorders; this finishes the sweep's output, with every remaining candidate opened rather than
named from the tool's summary.

**Five flipped, each needing a one-field seam change:** `createMeasure`, segment create, segment
UPDATE, segment delete, `uploadEvidence`. `CreateMeasureInput`, `CreateSegmentInput` and
`InsertEvidenceInput` now accept the value the event keys on — optional, minted by the store when
absent, so every other caller is unchanged. That was the whole obstacle: the store minted the id, or
for evidence the `uploadedAt` the payload reports as `payload.timestamp`.

Segment UPDATE needed three writes moved rather than one (`updateSegment`, `setMeasures`,
`setOverrides`), so a failure after the first left a partly-updated segment with no event at all; its
404 became an explicit pre-read, because `updateSegment` returning null WAS the not-found signal, which
is what made the old order unavoidable. `uploadEvidence` audits before the BUCKET write too — an object
in storage the ledger never mentions is harder to notice than a missing row.

**`src/audit/audit-order.test.ts` exists because nothing tested the rule.** Nine call sites had been
flipped across three commits and no test could tell: every existing test asserts the event EXISTS after
a SUCCEEDING operation, which is equally true in either order, so a reorder back was silent. Each case
makes the MUTATION fail and requires the event anyway — the only externally visible difference between
the orders. Mutation-checked on two.

Still mutate-first, with the reason now at each call site: the run-created transition and the
import-driven finalize (deliberate — the event is best-effort at the run boundary); `dispatchOutreach`,
which dispatches a message before any ledger entry and builds its payload from the delivery result, so
it needs ADR-073 d4's intent-then-completion pair and that adds an event type consumers read; the three
identity-link writes, whose obstacle is sharper than "the store mints the id" — `upsertLink` returns the
EXISTING row's id on conflict, so keying those events on the PAIR is the fix and it changes what
`entity_id` means; and the two backfill scripts, which are seeding tools rather than operator surfaces.

Checked and NOT violations, every one a matcher artifact: `audit-packet` (a hash), `materialize-run` and
`backfill-trend-history` (reads), evidence download, `measure-seed` (itself audit-first), subject-list
create (its audit is a `beforeComplete` callback that runs before the list becomes visible), and panel
assignment, which audits before the mapping and records each per-case event before `assignCases`.

The sweep is fully triaged, which is not #598 closing: what remains is the cross-store `applyCaseAction`
primitive plus the outreach and identity decisions. §4 and CLAUDE.md say exactly that.

Backend 2,856 tests: 2,832 pass, 23 skip, 1 pre-existing local failure (`corpus-membership`).

* fix(audit): the review's corrections — a 404 I dropped, and three tests that could not fail

Review of this branch found seven things. One is a regression this branch introduced; three are tests
asserting a weaker property than their titles claim.

**The PUT's relocated 404 dropped a guard that also protected the two later writes.** `updateSegment`
returning null was the not-found signal, and moving the 404 to an explicit pre-read discarded it — so a
row vanishing between the check and the write gave either a 500 (`setMeasures` violating the
`segment_measures` foreign key) or an HTTP 200 whose body is `null`, where the old order returned a
clean 404 for both. Two concurrent admin requests reach it. The return value is checked again, before
the child writes.

**All three new segments tests passed against the pre-change code.** The reason was a false claim in the
test file's own header: that the route's ordering is unreachable because it resolves its stores from
`env`. It is reachable — the store is a class, and patching its prototype makes a write fail against the
real fixture. The DELETE case asserted only that the payload name came from a pre-read, which was true
before the change too. Both cases now make the write fail and require the event to survive, and the PUT
gets one for the vanished-row 404.

**The event reported a measure list the row would never hold.** `setMeasures` writes `[...new Set(...)]`
and `hydrate` reads back ordered, so a payload built from the request array named something the segment
never contained. Harmless while the audit came second; a payload-accuracy regression once it comes
first, in the direction #598 exists to close.

**None of the three new seams was exercised by the store contract**, so the deployed Postgres ceiling was
asserted nowhere: deleting `input.id ??` from the SQLite adapter failed a test, and the identical edit to
the Pg adapter failed nothing. Three contract cases now, on both stores.

**Four existing audit-first paths had no ordering test** despite §4 saying one belongs —
`transitionStatus`, `createTerminologyMapping`, value-set attach and detach.

**§4's completeness claim was wrong for the third time.** `backfill-trend-history` was filed under "not a
violation (reads)" on the strength of two of its four hits; the other two are writes.
`recover-stuck-runs`, `resolve-valuesets` and `batch-evaluate-scale` were missing altogether, and the
PUT's own writes now surface as matcher artifacts against the DELETE's audit. Each time the prose was
plausible and the arithmetic was not done — so §4 now carries the count, 55 hits across 20 files, plus
the one-line command that re-derives it.

Recorded rather than fixed: EVIDENCE_UPLOADED now reaches the case TIMELINE (`audit_events WHERE
ref_case_id`), so a failed bucket write leaves a permanent "Evidence uploaded — <filename>" row with
nothing to download. The rule picks the over-claim side for the ledger; whether a clinical-ops read
surface should inherit it for a named file is an owner call, and §4 says so now.

Backend 2,863 tests: 2,839 pass, 23 skip, 1 pre-existing local failure. Mutations killed: reverting the
PUT ordering, dropping the null check, reverting the dedupe, and re-minting each of the three seam
values — each failing only its own case.

* fix(audit): SEGMENT_UPDATED reports what the request changes, not a guessed post-state (Codex, #612)

Merging the request over a pre-read produced a post-state, and under concurrency that post-state is a
guess: read `enabled: true`, let another admin set it false, change only the name, and `updateSegment`
preserves the newer false while the event reports true. The post-write hydration this replaced could
not be wrong about it, because it re-read — and an audit-first event cannot re-read.

So the payload is now what THIS REQUEST changes: each field the body supplies, plus a `changed` list
naming exactly that set, and nothing about the fields it does not set. `measureIds` stays deduped,
because that is what the row will hold. A consumer wanting the resulting state reads the row; what the
ledger is for is who changed what, and every value here is knowable before the write and true after it.

Pinned by a deterministic race rather than an argument: another writer flips `enabled` between this
request's pre-read and its write, and the assertions are that the event says nothing about `enabled`
and that the other writer's value survives — which is what the merged form would have mis-reported.

Codex's other two findings on this PR were already closed by the previous commit: rechecking
`updateSegment`'s result, and §4's incomplete audit-order triage.

Backend 2,864 tests: 2,840 pass, 23 skip, 1 pre-existing local failure. Mutation-checked: restoring the
merged payload fails the race case.

---------

Co-authored-by: Taleef <taleef@gmail.com>
Taleef7 added a commit that referenced this pull request Sep 23, 2026
…l) (#607)

* fix(qicore): preparation supplies a system, never a code — and the importer says what the document says

#594. The 2026-09-07 review's one high-priority correctness defect, accepted
2026-09-08 and untouched since.

prepareForQiCore filled four coded fields when it could not bind them:
Condition.clinicalStatus, verificationStatus, category and Encounter.class.
Its guard, unbindable(), is true of a MISSING field as well as a
present-but-unbindable one, and both branches assigned a module default.

Two consequences. First, an absent field was invented — live rather than
latent, because the QRDA-I import emits none of those three on a Condition
and no class on an Encounter, so preparation minted them on a third party's
document, including stamping `active` on a Condition the importer had just
given an abatementDateTime from a closed interval.

Second, and worse than the issue recorded: a PRESENT code was discarded.
unbindable() is as true of a system-less `resolved` as of a system-less
`active`, so a corrected misdiagnosis was reported as an active, confirmed
problem — that patient enters CMS122's denominator and, with no HbA1c, its
numerator. The file's own docstring claimed this hole was closed.

So: this layer supplies a SYSTEM, never a CODE. Normalize only when a value
is present, cannot bind, and carries a code from that field's own value set,
writing that same code back. Absent stays absent; bindable is untouched; an
unrecognised code is left alone, because we cannot claim to know which
system it came from.

The mapping moves to where the source semantics are known. times() now
reports three states rather than two, because collapsing them made a
faithful mapping impossible however it was written: a <high> with a value
closes the interval (resolved, beside its own abatementDateTime), a <high
nullFlavor="UNK"/> is QDM open prevalence (active), and an absent <high> is
silence and emits nothing.

Nothing the pilot reports moves: the ADR-075 corpus records all four fields
itself, fully systemed, with category distinguishing an encounter diagnosis
from a problem-list item — and cms122/125/2/137 still find real populations.

Fixtures are adversarial per the review's bar: refuted, resolved,
entered-in-error, an unrecognised code, an inpatient class, and the three
interval shapes. Four mutations killed, including "code discarded, default
substituted" — the first cut of this fix turned Encounter {code:"IMP"} into
ambulatory and an existing test caught it.

* fix(corpus): a bundle carries only what was known by its as-of

#595, and ADR-086 covering it with #594.

A corpus bundle built as of 2026-09-07 carried 19 future-dated clinical
events in the first 48 records — a PHQ-9 on 8 October, a blood pressure on
14 October, mammography in October and November. The bundle knew things that
had not happened.

Facts are generated for the calendar year the evaluation date falls in,
which is correct (ADR-072 scores a calendar year), and nothing then filtered
what was EMITTED by the as-of.

The cutoff filters on the date each fact was RECORDED — the value already
handed to provenanceFor — and not on "every date inside the resource is in
the past": a medication order known today may legitimately carry a future
intended end, and dropping it would be a different wrong answer. A resource
and its Provenance are emitted together, so a filtered fact leaves no
status, abatement, reference or provenance behind.

No reported number moves, and that is asserted rather than argued: every
official measurement is taken at 31 December, so the year-end cutoff is
compared against an unbounded one and must be identical. cms122, cms125,
cms2 and cms137 still find real populations. A separate test requires the
mid-year cutoff to actually remove something, so the no-foreknowledge sweep
cannot pass against a filter that does nothing.

It stops being invisible the moment anything evaluates at another date: a
mid-year rerun, a demo "as of today", an encounter-time evaluation (MM-4),
or an acceptance cohort built around a timing boundary.

* fix(qicore): a parse failure is not an assertion, and normalizing an entry keeps its neighbours

Both Codex findings on #606, and both are the same class as the defect the
PR removes — which is the part worth recording.

A malformed `<high value="20240230"/>` was read as an open interval. The
first cut asked "a <high> exists and produced no date", which is equally
true of a date the SOURCE asserted and this importer could not parse. It
reported `active`: a status the document never made, about a condition
whose end we simply failed to read, and one that can put the patient into a
measure population. It is now keyed on the nullFlavor attribute, and on any
spelling — NI, NA and ASKU all mean the source addressed the end and
recorded no value for it, so keying on UNK alone would read the rest as
silence. A parse failure is a third thing and says nothing.

Normalizing one `category` entry discarded the others. The first cut
flattened every entry's codings, picked one recognised code and assigned
the result AS the whole array, so a Condition carrying two categories kept
one and lost the other with its text and extensions. Each entry is now
normalized on its own, everything else it carries is preserved, and an
entry that cannot be normalized passes through unchanged — the same reason
an unrecognised code is left alone one level up.

Three separate cuts of this change have now re-committed the error it
exists to remove: substituting a default code for IMP, reading a parse
failure as an assertion, and replacing an array to normalize one of its
entries. Each was caught by a test rather than by review of the idea, which
is the argument for writing the adversarial fixture first. ADR-086 records
it.

Five tests added, three more mutations killed.

* docs(audit): the audit rule holds for operator actions, not for the run path (#598)

CLAUDE.md states "every state change writes audit_event — no exceptions".
The run path is an exception, and it was discoverable only by reading
run-pipeline.ts, which is the shape of claim DATA_MODEL_CONTRACTS exists to
stop.

The two paths make opposite trades on purpose and both stay. An operator
action records the event FIRST and then applies the patch (recordCaseEvent
makes the action row and the audit row one transaction), so a failure
between them leaves an action recorded but not applied — never an unaudited
state change. A run upserts the case FIRST and audits best-effort, so a
failure there leaves a state change applied but unaudited, accepted because
the alternative strands an otherwise-complete run as RUNNING after the case
was already mutated.

What is missing is the primitive, not the ordering — and it cannot live in
one store: the action and audit rows belong to CaseEventStore, which already
opens its own BEGIN/COMMIT, while the patch belongs to CaseStore, so
applyCaseAction({patch, action, audit}) needs a transaction seam spanning
both. Deferred deliberately: it wants local Postgres, since the SQLite floor
cannot catch Pg-only SQL. Until it exists, nothing should build operational
reliance on the ledger being complete for run-created transitions, and a
reconciliation job is not a substitute without durable operation identity,
an expected version, a deadline and a visible failure state.

Also #599's backend half: validateTests checks SHAPE, not outcomes, and its
docstring said so only by omission. Three tests pin the limitation rather
than the copy — a fixture naming a subject that exists nowhere passes, and
two fixtures asserting opposite outcomes for one subject both pass. When
execution lands, those tests fail, which forces the label and the behaviour
to move together.

* fix(studio): the fixtures row says what it checks, not what it implied (#599)

Studio rendered "Test Fixtures ✅" and blocked activation until it passed,
which reads as: the fixtures ran and the measure produced the expected
outcomes.

validateTests never executes anything. It checks the list is non-empty and
that each entry has a name, a subject, and an expectedOutcome in the allowed
set — so activation was gated by a control that could not fail on the thing
its label implied, which is the vacuous-guard shape this repo keeps finding.

The row now reads "Fixtures Well-Formed — present and well-formed; not
executed against the measure".

Relabel only. Executing them is the real fix and belongs with the next
Studio work; #599 carries it, and the backend tests in the previous commit
are what make the two move together — they pin the limitation, so
implementing execution fails them.

Blast radius was bounded: Studio authoring is hidden from the pilot's
CASE_MANAGER seats, so the misleading tick was in front of engineering users
only. That is why it was not urgent, not a reason it was fine.

* docs(audit): the correction was too broad too — the paths, swept rather than recalled

Codex on #607. The first cut said "operator actions audit first and cannot
lose the event": true of CASE actions, false of several other operator
surfaces. That is the same failure one level up, inside a change whose
entire subject is claims being wider than the thing behind them.

Enumerated by sweeping for the mutate-before-audit shape instead of
recalling it.

AUDIT FIRST, cannot produce an unaudited state change: every case action
(recordCaseEvent makes the action row and the audit row one transaction,
patch follows), rerun-to-verify's case patch, bulk assign and panel
backfill via the batch form.

MUTATE FIRST, can apply a change and lose the event: the run-created case
transition; the measure lifecycle (create :54, approve :69, deprecate :87,
transition :118); segment create (routes/segments.ts:172);
terminology-mapping create (value-set-governance.ts:335).

Only the run's ordering is a considered trade — the alternative strands an
otherwise-complete run as RUNNING after the case was already mutated. The
other six are simply the order they were written in, which splits #598 into
a cheap half (flip them; no seam needed) and the primitive that still wants
a cross-store transaction. Recorded on the issue.

case-rerun.ts looked like a violation and is not: its first mutation creates
a RUN row, and the case patch is explicitly after an audit-first
recordCaseEvent. Checked rather than assumed, in both directions.

The sweep covered admin, case, measure, quality, program, compliance and
routes and is a heuristic, so the list is what it found rather than proof of
completeness — stated as such.

---------

Co-authored-by: Taleef <taleef@gmail.com>
Taleef7 added a commit that referenced this pull request Sep 23, 2026
…e gets a test (#612)

* fix(audit): finish #598's triage — five more audit-first, and the rule gets a test

The owner's #598 decision was that a path which CAN audit before it mutates should. #607/#608 took the
plain reorders; this finishes the sweep's output, with every remaining candidate opened rather than
named from the tool's summary.

**Five flipped, each needing a one-field seam change:** `createMeasure`, segment create, segment
UPDATE, segment delete, `uploadEvidence`. `CreateMeasureInput`, `CreateSegmentInput` and
`InsertEvidenceInput` now accept the value the event keys on — optional, minted by the store when
absent, so every other caller is unchanged. That was the whole obstacle: the store minted the id, or
for evidence the `uploadedAt` the payload reports as `payload.timestamp`.

Segment UPDATE needed three writes moved rather than one (`updateSegment`, `setMeasures`,
`setOverrides`), so a failure after the first left a partly-updated segment with no event at all; its
404 became an explicit pre-read, because `updateSegment` returning null WAS the not-found signal, which
is what made the old order unavoidable. `uploadEvidence` audits before the BUCKET write too — an object
in storage the ledger never mentions is harder to notice than a missing row.

**`src/audit/audit-order.test.ts` exists because nothing tested the rule.** Nine call sites had been
flipped across three commits and no test could tell: every existing test asserts the event EXISTS after
a SUCCEEDING operation, which is equally true in either order, so a reorder back was silent. Each case
makes the MUTATION fail and requires the event anyway — the only externally visible difference between
the orders. Mutation-checked on two.

Still mutate-first, with the reason now at each call site: the run-created transition and the
import-driven finalize (deliberate — the event is best-effort at the run boundary); `dispatchOutreach`,
which dispatches a message before any ledger entry and builds its payload from the delivery result, so
it needs ADR-073 d4's intent-then-completion pair and that adds an event type consumers read; the three
identity-link writes, whose obstacle is sharper than "the store mints the id" — `upsertLink` returns the
EXISTING row's id on conflict, so keying those events on the PAIR is the fix and it changes what
`entity_id` means; and the two backfill scripts, which are seeding tools rather than operator surfaces.

Checked and NOT violations, every one a matcher artifact: `audit-packet` (a hash), `materialize-run` and
`backfill-trend-history` (reads), evidence download, `measure-seed` (itself audit-first), subject-list
create (its audit is a `beforeComplete` callback that runs before the list becomes visible), and panel
assignment, which audits before the mapping and records each per-case event before `assignCases`.

The sweep is fully triaged, which is not #598 closing: what remains is the cross-store `applyCaseAction`
primitive plus the outreach and identity decisions. §4 and CLAUDE.md say exactly that.

Backend 2,856 tests: 2,832 pass, 23 skip, 1 pre-existing local failure (`corpus-membership`).

* fix(audit): the review's corrections — a 404 I dropped, and three tests that could not fail

Review of this branch found seven things. One is a regression this branch introduced; three are tests
asserting a weaker property than their titles claim.

**The PUT's relocated 404 dropped a guard that also protected the two later writes.** `updateSegment`
returning null was the not-found signal, and moving the 404 to an explicit pre-read discarded it — so a
row vanishing between the check and the write gave either a 500 (`setMeasures` violating the
`segment_measures` foreign key) or an HTTP 200 whose body is `null`, where the old order returned a
clean 404 for both. Two concurrent admin requests reach it. The return value is checked again, before
the child writes.

**All three new segments tests passed against the pre-change code.** The reason was a false claim in the
test file's own header: that the route's ordering is unreachable because it resolves its stores from
`env`. It is reachable — the store is a class, and patching its prototype makes a write fail against the
real fixture. The DELETE case asserted only that the payload name came from a pre-read, which was true
before the change too. Both cases now make the write fail and require the event to survive, and the PUT
gets one for the vanished-row 404.

**The event reported a measure list the row would never hold.** `setMeasures` writes `[...new Set(...)]`
and `hydrate` reads back ordered, so a payload built from the request array named something the segment
never contained. Harmless while the audit came second; a payload-accuracy regression once it comes
first, in the direction #598 exists to close.

**None of the three new seams was exercised by the store contract**, so the deployed Postgres ceiling was
asserted nowhere: deleting `input.id ??` from the SQLite adapter failed a test, and the identical edit to
the Pg adapter failed nothing. Three contract cases now, on both stores.

**Four existing audit-first paths had no ordering test** despite §4 saying one belongs —
`transitionStatus`, `createTerminologyMapping`, value-set attach and detach.

**§4's completeness claim was wrong for the third time.** `backfill-trend-history` was filed under "not a
violation (reads)" on the strength of two of its four hits; the other two are writes.
`recover-stuck-runs`, `resolve-valuesets` and `batch-evaluate-scale` were missing altogether, and the
PUT's own writes now surface as matcher artifacts against the DELETE's audit. Each time the prose was
plausible and the arithmetic was not done — so §4 now carries the count, 55 hits across 20 files, plus
the one-line command that re-derives it.

Recorded rather than fixed: EVIDENCE_UPLOADED now reaches the case TIMELINE (`audit_events WHERE
ref_case_id`), so a failed bucket write leaves a permanent "Evidence uploaded — <filename>" row with
nothing to download. The rule picks the over-claim side for the ledger; whether a clinical-ops read
surface should inherit it for a named file is an owner call, and §4 says so now.

Backend 2,863 tests: 2,839 pass, 23 skip, 1 pre-existing local failure. Mutations killed: reverting the
PUT ordering, dropping the null check, reverting the dedupe, and re-minting each of the three seam
values — each failing only its own case.

* fix(audit): SEGMENT_UPDATED reports what the request changes, not a guessed post-state (Codex, #612)

Merging the request over a pre-read produced a post-state, and under concurrency that post-state is a
guess: read `enabled: true`, let another admin set it false, change only the name, and `updateSegment`
preserves the newer false while the event reports true. The post-write hydration this replaced could
not be wrong about it, because it re-read — and an audit-first event cannot re-read.

So the payload is now what THIS REQUEST changes: each field the body supplies, plus a `changed` list
naming exactly that set, and nothing about the fields it does not set. `measureIds` stays deduped,
because that is what the row will hold. A consumer wanting the resulting state reads the row; what the
ledger is for is who changed what, and every value here is knowable before the write and true after it.

Pinned by a deterministic race rather than an argument: another writer flips `enabled` between this
request's pre-read and its write, and the assertions are that the event says nothing about `enabled`
and that the other writer's value survives — which is what the merged form would have mis-reported.

Codex's other two findings on this PR were already closed by the previous commit: rechecking
`updateSegment`'s result, and §4's incomplete audit-order triage.

Backend 2,864 tests: 2,840 pass, 23 skip, 1 pre-existing local failure. Mutation-checked: restoring the
merged payload fails the race case.

---------

Co-authored-by: Taleef <taleef@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Studio's green tick on Test Fixtures implies they ran; validateTests never executes the measure

1 participant